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Detailed Action 



1 . This office action is in response to an AMENDMENT entered February 14, 2005 
for the patent application 09/875434 filed on June 5, 2001 . 



2. The First Office Action of November 14, 2005 is fully incorporated into this Final 
Office Action by reference. 



Status of Claims 

3 Claims 1 -1 3 are pending. 



Claim Rejections - 35 USC § 103 



4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 
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Claims 1 t 2, 3, 4 are rejected under 35 U.S.C. 103(a) as being unpatentable in 
view of Manjunath & Deng and further in view of Manjunath & Ma ('Issues for 
ImageA/ideo Digital Libraries', referred to as MD; Texture Features for Browsing 
and Retrieval of Image Data', referred to as MM) 

Claim 1. 

MD teaches (a) a task component configured to perform a plurality of 
classification tasks arranged in an established sequential progression of decision 
making (MD, p595, C2:10-1 1; Examiner's Note (EN) The first stage, segmentation 
search is the first of a plurality of classification tasks. This is the first part of a system 
decision system software.), said established sequential progression of decision making 
including a plurality of classification nodes for assigning class labels to an individual 
image file of said image files of non-textual subject data such that said class labels are 
available for matching a query when a search for said individual image file is 
subsequently conducted (MD, p595, C2:29 through p596, C1:10; EN 'Class labels' of 
applicant is equivalent to 'codeword' and corresponding 'icon representation' of MD.), at 
least some of said classification nodes including algorithms for determining which of a 
plurality of alternative next classification nodes is to be encountered in said sequential 
progression of decision making (MD, p595, C2.41-43; EN In this reference, a query 
pattern request will be used to find a match. This illustrates a plurality of next 
classification nodes to go to. MD p596 C1:1 1 to C2:14; MD uses a neural network along 
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with a vector quantization scheme to develop an hierarchical indexing which is 
equivalent to determining the next classification node of applicant.); 

MD teaches (b) an algorithmic component having access to a storage of 
available algorithms for execution at said classification nodes, said algorithmic 
component being common to said classification nodes and being accessed by each 
said classification node for selecting a specific algorithm for each of said classification 
tasks (MD, p395 C2:44 through p596. C1:6; EN Here the 'access to a storage of 
available algorithms' of applicant is equivalent to Visual thesaurus' of MD. The various 
levels of hierarchy of MD is equivalent to 'classification nodes being accessed by each 
said classification nodes' of applicant.), said specific algorithm being configured to 
execute at least one of content-based analysis for processing content-based data (MD, 
p596, C1:11 through p596, C2:5) 

MD does not teach meta-data analysis for processing meta-data. 

MM teaches meta-data analysis for processing meta-data (MM, p841, C1:18-20 
If pre analysis use of meta data is desired then MD p596 C2:37 through p597 C1 :3 
describes using an interface with icons that represent meta-data for queries.). It would 
have been obvious to a person having ordinary skill in the art at the time of applicant's 
invention to modify the method of determination of classification of MD with meta-data 
analysis for processing meta-data by MM. Having information about the non-textual file 
besides the content of the file itself is needed as well for classification purposes. 

MD teaches for at least some of said classification nodes said algorithmic 
component is configured to select among alternative stored algorithms that are specific 
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to determining assignment of a same said class label (MD, p596 t C2:28-31; EN After 
the image is segmented into regions by the first algorithm, alternative algorithms such 
as color, shape texture and location are possible used),said algorithmic component 
being further configured to use prior determinations at said classification nodes as a 
basis for selecting among said alternative stored algorithms specific to determining 
assignment of said same class label (MD, p597, C1:16 through C2:4; EN the 'prior 
determinations' of applicant is equivalent to 'sub-objects' in number of video frames in 
sequence.); 

MD teaches (c) a sub-algorithmic component for selecting at least one 
sub-algorithmic routine for said specific algorithm having a plurality of sub-algorithm 
routines, said at least one sub-algorithmic routine being selected based on said 
selecting said algorithm (MD, p597, C2:5-16; EN Sub-object information is now 
generated and using that, sub-algorithms of spatial and temporal relations are now 
used.); and 

MD teaches (d) a learning component for modifying said arrangement of 
classification tasks according to determinations of frequency patterns In the common 
assignments of said .class labels to individual said image files of non-textual subject 
data(MD, p596, C2:5-9; EN A hybrid neural network is a design that learns from 
exposure to frequency of patterns). 



Claim 2. 
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MD teaches a system web-service module for providing Internet access to said 
system decision module (MD, p596 C2:41 through p597 C1:2). 

Claim 3. 

MD teaches a system interface module for providing communications among a 
plurality of system and nonsystem modules, wherein one of said system modules Is 
said system decision module (MD, p596, C2:37-41). 

Claim 4. 

MD teaches non-system modules includes at least one said sub-algorithmic 
routine (MD, p597, C1:5-7; EN Non-system modules such as object tracking scheme 
would have working algorithm(s) to make them functional. 



Claim Rejections - 35 (JSC § 103 

5. Claims 5, 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of MD, and MM, as set forth above, and further in view of Kadtke (U. S. 
Patent 6278961 , referred to as Kadtke). 

Claim 5. 

MD and MM do not teach the system interface module further includes data 
components for storing data associated with classifying a plurality of said image files of 



Application/Control Number: 09/875,434 Page 7 

Art Unit: 2129 

said non-textual subject data and at least one control component for executing said 
sub-algorithmic routines. 

Kadtke teaches the system wherein said system interface module (Kadtke, 
Figure 2; EN The 'system interface' of applicant enables communication between 
modules. In Figure 3, you can see the communication between modules 201 , 202, 203, 
204 and 205 of Kadtke.) further includes data components for storing data associated 
with classifying a plurality of said image files of said non-textual subject data (Kadtke, 
C2:44-47; EN The 'storing data' of applicant is equivalent to 'build and modify a 
database of features' of Kadtke.) and at least one control component for executing said 
sub-algorithmic routines (Kadtke, Figure 1; EN Kadtke shows in control component 
105, has three different sub-algorithms to choose from.). It would have been obvious to 
a person having ordinary skill in the art at the time of applicant's invention of a system 
for classifying files of non-textual subject data of MD and MM with the system wherein 
said system interface module further includes data components for storing data 
associated with classifying a plurality of said image files of said non-textual subject data 
and at least one control component for executing said sub-algorithmic routines by 
Kadtke. This allows for ease of communication of new modules if added and little effect 
if removed, having a storage facility with images and associated classification for ease 
of retrieval, and easing system design by having multiple sub-algorithms associated to a 
single node. 



Claim 7. 
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MD and MM do not teach the learning component is configured to identify an 
algorithm for each of said classification tasks and at least one sub-algorithmic routine 
for said algorithm. 

Kadtke teaches the learning component is configured to identify an algorithm for 
each of said classification tasks and at least one sub-algorithmic routine for said 
algorithm (Kadtke, C7:66 through C8:5; EN the learning component of applicant is 
equivalent to 'neural nets' of Kadtke. A neural network can take in data from and 
algorithm and generate the correct sub-algorithm to use for the next classification step. 
It would have been obvious to a person having ordinary skill in the art at the time of 
applicant's invention of a system for classifying files of non-textual subject data of MD 
and MM with teaches the system wherein said learning component is configured to 
identify an algorithm for each of said classification tasks and at least one 
sub-algorithmic routine for said algorithm by Kadtke. Using a neural net for this has the 
advantages of learning and dynamically changing which sub-algorithm to choose from 
given the inputted data. 

Claim Rejections - 35 USC § 103 

6. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of MD, and MM, as set forth above, and further in view of Melen (U. S. 
Patent 5719960, referred to as Melen). 
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Claim 6. 

MD and MM do not teach a media input/output module for administering data 
associated with classifying said non-textual subject data by reading and writing said 
data among a plurality of modules. 

Melen teaches a media input/output module for administering data associated 
with classifying said non-textual subject data by reading and writing said data among a 
plurality of modules (Melen, C7:15-18). It would have been obvious to a person having 
ordinary skill in the art at the time of applicant's invention of a system for classifying files 
of non-textual subject data of MD and MM with the system further comprising a media 
input/output module for administering data associated with classifying said non-textual 
subject data by reading and writing said data among a plurality of modules by Melen. 
This allows for faster and more secure data input. 

Claim Rejections - 35 USC § 103 

7. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of MD, and MM, as set forth above, and further in view of Wagner (U. S. 
Patent 5950180, referred to as Wagner). 

Claim 8. 

MD and MM do not teach the data capturing device configured to capture said 
content-based data and record said meta-data, said content-based data corresponding 
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to content information of a file of said subject data and said meta-data corresponding to 
situational environmental data of said data capturing device during a capture of said 
subject data. 

Wagner teaches the data capturing device configured to capture said 
content-based data and record said meta-data, said content-based data corresponding 
to content information of a file of said subject data and said meta-data corresponding to 
situational environmental data of said data capturing device during a capture of said 
subject data (Wagner, C4:10-12). It would have been obvious to a person having 
ordinary skill in the art at the time of applicant's invention of a system for classifying files 
of non-textual subject data of MD and MM with the system of claim 1 further comprising 
a data capturing device configured to capture said content-based data and record said 
meta-data, said content-based data corresponding to content information of a file of said 
subject data and said meta-data corresponding to situational environmental data of said 
data capturing device during a capture of said subject data by Wagner. This permits 
input to come from the real world thus having the ability to work in today's situation. 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
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subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 

Claims 9, 10 t 11 and 12 are rejected under 35 U.S.C. 103(a) as being 
unpatentable in view of Namjunath & Deng and further in view of Wagner and 
further in view of Kadtke. ('Issues for ImageA/ideo Digital Libraries', referred to as 
MD; U. S. Patent 5950180, referred to as Wagner; U. S. Patent 6278961, 
referred to as Kadtke) 



Claim 9. 

MD teaches establishing a sequential progression of decision making, including 
using automated processing techniques to define a dependent arrangement of a 
plurality of task nodes, each said task node in said dependent arrangement being 
associated with a class label for classifying a data file (MD, 595, C2:9-14; EN 
Segmentation is the first step of an established progression of decision making. Region 
based search techniques are the next step with this design. Region based is dependent 
of segmentation and they are within their own separate classes of algorithms), at least 
some of said task nodes including algorithms for determining which alternative next task 
node is to be selected in said sequential progression of decision making, said task 
nodes including multi-algorithmic task nodes having a plurality of alternative said 
algorithms for implementing said determination, each said multi-algorithmic task node 
being specific to determining assignment of a particular said class label for 
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availability in matching a query during a subsequent search and each said alternative 
algorithm at said multi-algorithmic task node being specific to said particular class 
label;(MD, p595, C2:29 through p596, C1:10; EN 'Class labels 1 of applicant is equivalent 
to 'codeword' and corresponding 'icon representation' of MD. MD, p595, C2: 15-20; EN 
As stated above, the region based search follows a sequential pattern for decision in 
classification. In region based search there are a number of areas which the method 
can choose from. Texture, color, location are but a few multi-algorithms not within the 
same class which the task node can pick from.) 

MD does not teach receiving a file of non-textual subject data, 
Wagner teaches receiving a file of non-textual subject data (Wagner, C4:10-12). 
It would have been obvious to a person having ordinary skill in the art at the time of 
applicant's invention of a system for classifying files of non-textual subject data of MD 
with receiving a file of non-textual subject data by Wagner. Using a CCD camera is a 
standard method for an image capturing device. 

MD and Wagner do not teach progressing said file through said dependent 
arrangement defined in said establishing said sequential progression of decision 
making. 

Kadtke teaches progressing said file through said dependent arrangement 
defined in said establishing said sequential progression of decision making (Kadtke, 
Figure 2; EN This shows a flow chart that is sequential and containing dependent 
arranged modules). It would have been obvious to a person having ordinary skill in the 
art at the time of applicant's invention of a system for classifying files of non-textual 
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subject data of MD and Wagner with progressing said file through said dependent 
arrangement defined in said establishing said sequential progression of decision making 
by Kadtke. By using a standard consistence approach, this allows for modifications to 
be made with little effect on other modules. 

MD and Wagner do not teach selecting from among said alternative algorithms at 
said multi-algorithmic task nodes. 

Kadtke teaches selecting from among said alternative algorithms at said 
multi-algorithmic task nodes ( Kadtke, Figure 1 ). It would have been obvious to a 
person having ordinary skill in the art at the time of applicant's invention of a system for 
classifying files of non-textual subject data of MD and Wagner with selecting from 
among said alternative algorithms at said multi-algorithmic task nodes by Kadtke. This 
allows for a simpler design when constructing the decision making process. 

MD and Wagner do not teach utilizing an algorithmic component to perform said 
selection, said selection at least partially based on prior determinations at previously 
encountered task nodes in said sequential progression. 

Kadtke teaches utilizing an algorithmic component to perform said selection, said 
selection at least partially based on prior determinations at previously encountered task 
nodes in said sequential progression (Kadtke, C9:25-29). It would have been obvious 
to a person having ordinary skill in the art at the time of applicant's invention of a system 
for classifying files of non-textual subject data of MD and Wagner with utilizing an 
algorithmic component to perform said selection, said selection at least partially based 
on prior determinations at previously encountered task nodes in said sequential 
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progression by Kadtke. By using previous encountered task nodes and statically 
methods, the performance of the method can be improved upon. 

Claim 10. 

MD teaches a learning procedure in which content-based data is extracted from 
each of a plurality of training images and meta-data is identified for each said training 
image (MD, p596, C2:34-35). 

Claim 11. 

MD does not teach the step of generating a plurality of learning classes that are 
descriptive of said training Images, including using an association pattern technique of 
recognizing and using patterns In assignments of said class labels, said step of 
generating including applying content-based analysis for said content-based data and 
meta-data analysis for said meta-data. 

Wagner teaches a step of generating a plurality of learning classes that are 
descriptive of said training Images, including using an association pattern technique of 
recognizing and using patterns In assignments of said class labels, said step of 
generating including applying content-based analysis for said content-based data and 
meta-data analysis for said meta-data (Wagner, C5:36-39). It would have been obvious 
to a person having ordinary skill in the art at the time of applicant's invention of a system 
for classifying files of non-textual subject data of MD with generating a plurality of 
learning classes that are descriptive of said training Images, including using an 
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association pattern technique of recognizing and using patterns In assignments of said 
class labels, said step of generating including applying content-based analysis for said 
content-based data and meta-data analysis for said meta-data by Wagner. EN By using 
a neural network, associating patterns and recognizing said patterns is easily 
accomplished. 

Claim 12. 

MD does not teach the step of dynamically modifying said sequential progression 
of decision making, including monitoring said determinations at each of said decision 
nodes and adjusting for detected patterns in said determinations. 

Wagner teaches the step of dynamically modifying said sequential progression of 
decision making, including monitoring said determinations at each of said decision 
nodes and adjusting for detected patterns in said determinations (Wagner, Figure 4; EN 
By using a neural network, dynamically modifying the decision making progress is 
already there in the design.). It would have been obvious to a person having ordinary 
skill in the art at the time of applicant's invention of a system for classifying files of non- 
textual subject data of MD with a step of dynamically modifying said sequential 
progression of decision making, including monitoring said determinations at each of said 
decision nodes and adjusting for detected patterns in said determinations by Wagner. 
Having the ability to dynamically alter the decision making progress, will have the effect 
of a more efficient machine and the ability to change with the current environment. 
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Claim Rejections - 35 USC § 103 

9. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of MD, Wagner and Kadtde, as set forth above, and further in view of Shu 
(U. S. Patent 4901360, referred to as Shu). 

Claim 13. 

MD, Wagner and Kadtke do not teach a step of assigning a semantic description 
to said file of non-textual subject data for one of organizing said file and matching a 
query during a search for said file. 

Shu teaches a step of assigning a semantic description to said file of non-textual 
subject data for one of organizing said file and matching a query during a search for 
said file (Shu, C14:1-4). It would have been obvious to a person having ordinary skill in 
the art at the time of applicant's invention of a system for classifying files of non-textual 
subject data of MD, Wagner and Kadtke with a step of assigning a semantic description 
to said file of non-textual subject data for one of organizing said file and matching a 
query during a search for said file by Shu. This will act as another link between the user 
and the file for organizing purposes. 



Response to Arguments 
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10. Applicant's arguments filed on Feburary 14,2006 for claims 1-1 3 have been fully 
considered but are not persuasive. 



1 1 . Applicant amended the claims to overcome the 35 U. S. C. §101 objection. 
Examiner withdraws the §101 objection. 



12. In reference to the Applicant's argument: 

In response to the rejection of the claims, independent claims 1 and 9 have been amended 
to more clearly distinguish the claimed invention from the cited prior art. As amended, 
claim 1 describes the system decision module as including a task component 
configured to perform classification tasks, wherein an established sequential 
progression of decision making includes classification nodes for assigning class labels 
to an individual image file such that the class labels are available for matching a query 
when a search for the individual image file is subsequently conducted. The amendment 
to claim 1 provides clarity with regard to the invention being used to assign the class 
labels. That is, the components described in claim 1 are cooperative with regard to 
assignment of the class labels, which can then be used subsequently in conducting a 
search. Support for the amendment to claim 1 may be found on page 3, lines 14-16 of 
the application as originally filed. The application states that the class labels that are 
selected as descriptors of a particular image are utilized for organization and "for 
matching a query when a search for the image is subsequently conducted." 

Examiner's response: 

MD p595 C2:29 through p596 C1 :10 describes a visual thesaurus in which 

features are computed. These features are clustered into groups of a particular 

feature. These groups are related to a codeword, which has it's own visual icon. 

'Labels that are available for matching a query' of applicant is equivalent to 'visual 



icons' of MD. First Office Action applies. 
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1 3. In reference to the Applicant's argument: 

The amendment to claim 9 is similar to that of independent claim 1 . The computer 
implemented method is now described as including using automated processing 
techniques to define a dependent arrangement of task nodes that Include multi- 
algorithmic task nodes. Each multi-algorithmic task node is specific to determining 
assignment of a particular class label for availability jn matching query in a subsequent 
search. Support for the amendment to claim 9 is the same as the support for the 
amendment to claim 1 . 

Examiner's response: 

MD p596 C2:37 through p597 C1:3 describes a user interface. The user can 

select color, object texture. These query options represent searching algorithms that 

exists in MD. 'Multi-class algorithmic node' of applicant is equivalent to 'query interface' 

of MD. First Office Action applies. 



14. In reference to the Applicant's argument: 

By amending claims 1 and 9 to describe the elements of the invention as being 
cooperative in assigning class labels, the invention is distinguished from components and 
method steps that are Implemented to take advantage of class labels which have been 
previously assigned. In the determination of patentability of the claims prior to 
amendment, the teachings of the prior art with regard to assigning °class labels" were 
merged with the teachings of the prior art with regard to use of the class labels which 
were previously assigned. For example, in the rejection of claim 1 , it is agreed that MD 
does not teach meta-data analysis for processing meta-data, but it is then noted that the 
secondary reference to MM teaches the use of meta-data on page 841 , column 1 , lines 1 8- 
20. However, the meta-data described in this portion of MM is specific to the "class 
labels" themselves (i.e., the texture features) that have been extracted from the original 
image. Claim I describes the use of meta-data as being a means to achieve a result, 
whereas the meta-data of MM is the final result in the assignment of class labels. 
Similarly, the teachings of the primary reference to MD with respect to post-label 
assignment" queries are cited in the rejection of the claims. For example, it is noted in 
an Examiner's Note (EN) that column 2, lines 41-43 on page 595 of MD teach that a 
query pattern request will be used to find a match. 

Applicants respectfully request reconsideration of the claims in view of the 
amendments to the claims. 
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Examiner's response: 

If applicant saw the use of 'meta-data' as an analysis tool, then MD provides an 
example of this. MD p596 C2:37 through p597 C1 :3 describes using an interface with 
icons that represent meta-data for queries. First Office Action applies. 



1 5. In reference to the Applicant's argument: 

As previously noted, one difference between the amended claim and the 
teachings of the primary reference to MD is that claim I describes the class labels that 
are to be assigned as being available for matching a query .when a search for the 
individual image file is subsequently conducted. The query pattern request identified in 
the Office action occurs after an image in a database has been assigned its proper 
class labels. Moreover, the cited portions of column 2 on page 595 of MD do not teach a 
sequential progression in which at least some of the classification nodes include 
algorithms for determining which of a plurality of alternative next classification nodes is 
to be encountered within the sequential progression of decision making. The search for 
a match between assigned codewords is distinguishable from the decision making 
regarding whether the assignment of the class labels is to be made. 

With regard to component (b) in claim 1 (i.e., the algorithmic component), there is 
citation to the paragraph of.the primary reference to MD that begins on page 595 and 
ends on page 596. This paragraph merely states that the long term goal is to construct a 
visual thesaurus for images/video, where the codewords of the thesaurus are created at 
various levels of visual hierarchy by grouping primitives such as texture, color, shape and 
motion. Applicants respectfully assert that this does not teach or suggest the algorithmic 
component that is common to the classification nodes and that is accessed by each 
classification node for selecting a specific algorithm for each of the classification tasks. 
The primary reference describes examples of the codewords for texture as being 
"parking lots," airport tarmacs," "building developments," "vegetation patterns," and 
"highways." Fig. 1 In MD is an example of a region-based retrieval of parking lot areas 
in aerial photographs. The different codewords are the results of the application of 
algorithms, rather than being algorithms themselves. 

Examiner's response: 
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MD p596 C1 :1 1 to C2:14 uses a neural network along with a vector quantization 
scheme to develop an hierarchical indexing which is equivalent to determining the next 
classification node of applicant. If there exists an hierarchical structure then there is a 
determination point in which the next classification point would follow. 'Algorithmic 
component' of applicant is equivalent to 'neural network' of MD. First Office Action 
applies. 



16. In reference to the Applicant's argument: 

The Office action notes that MD does not teach meta-data analysis for processing meta- 
data. Therefore, the secondary reference to MM was cited. However, the reference to 
meta-data in column 1 on page 541 Is the stored results of the process, rather than the 
use of meta-data to achieve this result for storage. 



Examiner's response: 

MD teaches the use of an interface using icons that represent meta-data. This 
illustrates the use of meta-data to achieve results. First Office Action applies. 



1 7. In reference to the Applicant's argument: 

Also in component (b) of claim I is the description of at least some of the classification 
nodes in the sequential progression of decision making being configured to select 
among alternative stored algorithms that are specific to determining assignment of a 
same class label. The Office action asserts that this is taught by MD, since the primary 
reference refers to identification of color, shape, texture and location. Applicants 
respectfully point out that the alternative stored algorithms of claim 1 are specific to 
determining assignment of a game class label. Therefore, teachings regarding different 
class labels (e.g., color, shape and texture) do not read on claim 1. Additionally, the 
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alternative algorithms of claim 1 relate to a particular classification node, whereas a 
person of ordinary skill In the art would consider classification nodes relating to color, 
shape and texture as being different classification nodes. 

Component (b) of claim 1 identifies selection from alternative stored algorithms that are 
specific to determining assignment of a same class label, with prior determinations at 
classification nodes being used as the basis for selecting among the alternative 
algorithms specific to determining assignment of "said same class label." Then, in 
component (c) (i.e., the sub-algorithmic component), at least one sub-algorithmic 
routine is selected based on the selecting of the algorithm. With respect to component 
(c), the Office action cites column 2, lines 5-16 on page 597 of the primary reference to 
MD. This portion of the reference refers to the extension of the idea to videos. Within the 
given number of frames of a video, one frame is chosen for spatial segmentation. The 
regions obtained as a result of this segmentation are the ones that are tracked over the 
entire group of frames. Fig. 4 of MD is an example of region tracking over several groups 
of frames. Each video shot is thus composed of a set of subobjects and the video shot can 
be characterized by Its subobject information and the spatial and temporal relation between 
these subobjects. The representation allows a user to track regions in a video 
sequence and search for regions with similar color, texture, shape, motion pattern, location, 
or size. It is respectfully asserted that this neither teaches nor suggests the selection of sub- 
algorithmic routines for a specific algorithm selected from among alternative stored 
algorithms that are specific to deter-mining assignment of a same class label,. The 
composition of a set of subobjects from a video shot does not assign a class label. 

Examiner's response: 

MD illustrates determining differences within the same class (MD p596 C1 :1 1 

through C2:14) Under texture, MD shows examples of parking lots, airport tarmacs, 

building development, vegetation. There exists different algorithms for such 

classifications and all are within the same class of 'texture 1 and parking lots, airport 

tarmacs, building development, vegetation are equivalent to subobjects. First Office 

Action applies. 



18. 



In reference to the Applicant's argument: 
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With respect to the patentability of claim 9, the Office action notes that the primary 
reference to MD teaches segmentation as a first step of the process and teaches region- 
based search techniques as the next step. It is submitted in the Office action that these two 
steps of MD teach "establishing a sequential progression of decision making, including using 
automated processing techniques to define a dependent arrangement of a plurality of task 
nodes, each said task node in said dependent arrangement being associated with a class 
label for classifying a data file." Applicants respectfully point out that the segmentation is not 
associated with a class label for classifying a data file. This first step of MD merely defines 
segments. Claim 9 describes a sequential progression of decision making. Task nodes 
include multi-algorithmic task nodes having a plurality of alternative algorithms for 
implementing the determination. It is submitted in the Office action that texture, color and 
location are but a few multi-algorithms not within the same class from which the task node 
of MD can pick. However, Applicants respect-fully submit that a person of ordinary skill in 
the art would not interpret MD to teach a sequential progression of decision making in 
which the system decides to categorize based upon either texture, color or location. The 
reference does not appear to allow the system to select which features will be of interest. 
Instead, the sequential progression as taught by MD would be one in which the different 
types of features are considered in progression, if there is an established sequential 
progression as described in pending claim 9. 



Examiner's response: 

Incoming data files most likely will have more than one characteristic/class within 
them. The first step is to separate these classes and then classify them. Separating 
the incoming data file into its components is needed to fulfill a classification process. 
MD also illustrates classification with the same class as in texture. MD teaches a 
hierarchical image thesaurus which is equivalent to 'sequential progressive of decision 
making' of applicant. (MD p596 C1:11 through C2:14) First Office Action applies. 



1 9. In reference to the Applicant's argument: 

The Office action notes that neither the primary reference to MD nor the secondary 
reference to MM teaches progressing a file through a dependent arrangement defined 
in said establishing said sequential progression of decision making. Therefore, Kadtke 
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was cited. However, the fixed arrangement of Kadtke does not teach or suggest 
modifying the primary reference to MD such that at least some of the task nodes include 
algorithms for determining which alternative next task node is to be selected in the 
sequential progression of decision making. 

Kadtke was also cited for teaching selecting from alternative algorithms at multi-algorithmic 
task nodes. It is asserted that it would be obvious to modify MD so as to include the 
multi-algorithmic task nodes taught by Kadtke, since this allows a simpler design when 
constructing the decision making process. However, applicants assert that modifying 
MD to include the teachings of Kadtke would greatly increase the complexity. As noted with 
regard to the patentability of claim 1 , the first stated object of Kadtke is to provide a 
"theoretically well-founded method of signal processing and time series analysis which can 
be used in a variety of applications (such as Sonar, Radar, Lidar, seismic, acoustic, 
electromagnetic and optic data analysis) where deterministic signals are desired to be 
detected and classified." Thus, as noted in the ABSTRACT of Kadtke, the techniques 
use dynamic filters and classifiers optimized for a particular category of signals of interest. 
This capability and complexity is not advantageous to the system and method taught in 
MD. 

Examiner's response: 

MD p596 C1:1 1 through C2:14, teaches a hierarchical algorithm to which illustrates 
decision points of classification. First Office Action applies. 



Examination Considerations 

20. The claims and only the claims form the metes and bounds of the invention. 
"Office personnel are to give the claims their broadest reasonable interpretation in light 
of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44USPQ2d 1023, 
1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited in 
the claim are not read into the claim. In re Prater, 415 F.2d, 1393, 1404-05, 162 USPQ 
541,550-551 (CCPA 1969)" (MPEP p 2100-8, c2, I 45-48; p 2100-9, c 1, 11-4). The 
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Examiner has the full latitude to interpret each claim in the broadest reasonable sense. 
Examiner will reference prior art using terminology familiar to one of ordinary skill in the 
art. Such an approach is broad in concept and can be either explicit or implicit in 
meaning. 

21 . Examiner's Notes are provided to assist the applicant to better understand the 
nature of the prior art, application of such prior art and, as appropriate, to further 
indicate other prior art that maybe applied in other office actions. Such comments are 
entirely consistent with the intent and sprit of compact prosecution. However, and 
unless otherwise stated, the Examiner's Notes are not prior art but link to prior art that 
one of ordinary skill in the art would find inherently appropriate. 

22. Examiner's Opinion: Paragraphs 21 and 20 apply. The Examiner has full 
latitude to interpret each claim in the broadest reasonable sense. 



Conclusion 

23. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

24. Claims 1 -1 3 are rejected. 



Correspondence Information 

25. Any inquiry concerning this information or related to the subject disclosure should 
be directed to the Examiner Peter Coughlan, whose telephone number is (571) 272- 
5990. The Examiner can be reached on Monday through Friday from 7:15 a.m. to 3:45 
p.m. 

If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor David Vincent can be reached at (571) 272-3687. Any response to this 
office action should be mailed to: 

Commissioner of Patents and Trademarks, 
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Washington, D. C. 20231; 
Hand delivered to: 
Receptionist, 

Customer Service Window, 
Randolph Building, 
401 Dulany Street, 
Alexandria, Virginia 22313, 

(located on the first floor of the south side of the Randolph Building); 
or faxed to: 

(571 ) 273-8300 (for formal communications intended for entry.) 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://pair-direct.uspto.gov . Should you have any questions 
on access to Private PAIR system, contact the Electronic Business Center (EBC) at 
866-217-9197 (toll free). 
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